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Abstract 

A device has two arms with unknown deterministic payoffs, and 
the aim is to asymptotically identify the best one without spending too 
much time on the other. The Narendra algorithm offers a stochastic 
procedure to this end. We show under weak ergodic assumptions 
on these deterministic payoffs that the procedure eventually chooses 
the best arm (i.e. with greatest Cesaro limit) with probability one, 
for appropriate step sequences of the algorithm. In the case of i.i.d. 
payoffs, this implies a "quenched" version of the "annealed" result 
of Lamberton, Pages and Tarres in 2004 [6] by the law of iterated 
logarithm, thus generalizing it. 

More precisely, if (%,j)ieN £ {0, 1} N , I £ {A,B}, are the deter- 
ministic reward sequences we would get if we played at time i, we 
obtain infallibility with the same assumption on nonincreasing step se- 
quences on the payoffs as in [6], replacing the i.i.d. assumption by the 
hypothesis that the empirical averages Ysi=l r lA,i/ n arm Ya^iVb,*/ 71 
converge, as n tends to infinity, respectively to 6a and 8b, with rate 
at least l/(logn) 1+£ , for some e > 0. 
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1 Introduction 



1.1 General introduction 

The so-called two-armed bandit is a device with two arms, each one 
yielding an outcome in {0, 1} at each time step, irrespective of the 
strategy of the player, who faces the challenge of choosing the best 
one without loosing too much time on the other. 

The Narendra algorithm is a stochastic procedure devised to this 
end, which was initially introduced by Norman, Shapiro and Naren- 
dra [P2] in the fields of mathematical psychology and learning 
automata. An application to optimal adaptive asset allocation in a 
financial context has been developped by Niang [10J. 

Formally, let (fi, J 7 , P) be a probability space. The Narendra two- 
armed bandit algorithm is defined as follows. At each time step n G N, 
we play source A (resp. source B) with probability X n (resp. 1 —X n ), 
where Xq = x £ (0, 1) is fixed and X n is updated according to the 
following rule, for all n 0: 

IX n + 7„+i(l - X n ) if U n+ i = A and rj A , n +l = 1 
(l-j n+ i)X n if U n+ i = B and rj B , n +i = 1 0-) 
X n otherwise, 

where (7 n )n^i is a deterministic sequence taking values in (0, 1), U n+ \ 
is the random variable corresponding to the label of the arm played 
at time n + and f?A,n+i (resp. i]B, n +i) denotes the payoff, taking 
values in {0, 1}, of source A (resp. source B) at time n + 1. 

We assume without loss of generality that U n +i — A l{i n+1 ^.x n } + 
B !{7 n+ i>x n }> where (I n )n^i is a sequence of independent uniformly 
distributed random variables on [0, 1]. 

The literature on this algorithm generally assumes that the se- 
quences (r]A,n)n^i and (r]B,n)n^i are independent with Bernoulli dis- 
tributions of parameters 9a and 0b, where 9a > #b, the aim being to 
determine whether (X n ) nG ^ a.s. converges to 1 or not as n tends to 
infinity. 

Notwithstanding the apparent simplicity of this stochastic proce- 
dure, the first criteria on a.s. convergence to "the good arm" under 
the above i.i.d. assumptions were only obtained thirty years after the 
original definition of this Narendra algorithm, by Tarres [13] , and Lam- 
berton, Pages and Tarres [6] in a more general framework. Recently 
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Lamberton and Pages established the corresponding rate of conver- 
gence [I], and proposed and studied a penalized version [5j. Note 
that a game theoretical question arising in the context of two-armed 
bandits was recently studied by Benai'm and Ben Arous [lj. 

Our work focuses on the understanding of the Narendra two-armed 
bandit algorithm under the assumption that the payoff sequences 
(r]A,n)n^i, £ € {A,B}, are unknown and deterministic. Under the 
following condition (S) on the step sequence (required in [6], but 
without monotonicity) , and weak ergodic assumption (E2) empha- 
sizing the rate at which A must be asymptotically better than B, we 
show that X n a.s. converges to 1. Heuristically, the result points out 
that, even with strongly dependent outcomes, X n accumulates suffi- 
cient statistical information on the ergodic behaviour of the two arms 
to induce a corresponding appropriate decision. 

More precisely, let us introduce the following step sequence and 
ergodic assumptions. 

n 

Step sequence Conditions. Let, for all n E N, T n = ^^7fc- 

k=l 

Let (SI) and (S2) be the following assumptions on the step se- 
quence (7„)„ 6 n: 

(51) (7 n ) n >i is nonincreasing and T n — > oo; 

ri— >oo 

(52) 7 n = 0(r„e-^ r »). 

Let (S) be the set of conditions (S1)-(S2). 

Ergodic Conditions. Let (E) be the assumption that the ouputs of 
arms A and B satisfy 

.. n \ n 

(E) -}nA,k — > 9a, and -}VB,k — > 9 B , 

fl L — ' n— >oo n n—>oo 

k=l k=l 

where 6a, 9b € (0,1). The ergodic condition (E) means that the 
average payoff of arm A (resp. arm B) is 9a (resp. 9b), but does not 
assume anything on the corresponding rate of convergence. In order 
to introduce conditions on this rate, let us denote, for all n € N, 

n 

^2(ve,i-9e) ■ 

i=l 

Given a map 4> : N — > M + and 9a, 9b G (0, 1), let us denote by 
(E</>) the assumption that R n /(p(n) — ► 0. 

n— >oo 



R n := max 

ee{A,B} 
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Let (El) and (E2) be condition (E0), respectively with the fol- 
lowing assumption on (j>: 

(El) (f) is nondecreasing concave on [/cq,oo) for some k$ £ N, and 
sup n6N 7n0(n) < oo. 

(E2) 6(n) = — — \TT~i — for some e > 0. 

V (log(n + 2))!+ £ 

Note that (E) corresponds to (E</>) with 4>(n) = n, n G N, for 
which (El) holds for instance in the case of a step sequence j n = 
c/(c + n), c > 0. Also, the following Lemma [H proved in Section 
implies that (S)-(E2) (El). 

Lemma 1 If condition (S) holds, then limsup n ^ oc 7 ra n/ log n ^ 1/#_b 
and lim sup ra ^ oc T n / log n ^ 1 /9b ■ 

The following Theorems [2] and [3] provide assumptions for conver- 
gence of the Narendra sequence (X n ) n ^o towards or 1 as n tends 
to infinity, respectively convergence towards 1 when 9a > 9b (i-e. 
asymptotic choice of the "right arm"). 

Theorem 2 Under assumptions (Sl)-(El), the Narendra sequence 
(X n ) ng pj converges ¥ x — a.s towards or 1 as n tends to infinity. 

Theorem 3 Under assumptions (S)-(E2) and 9a > 9b, the Naren- 
dra sequence (X n ) ng ^ converges ¥ x — a.s towards 1 as n tends to 
infinity. 

Recall that the above conditions (El) and (E2) are purely de- 
terministic. If we let the sequences {r)A,i)i&* and (r/B,i)igN be dis- 
tributed as i.i.d. sequences with expectations 9a and 9b, then (E2) 
almost surely occurs as a consequence of the law of iterated logarithm. 
Assuming (S) and 9a > 9b, Theorem [3] implies that the algorithm 
(X n ) n £fi} almost surely converges to 1, which generalizes the corre- 
sponding infallibility Proposition 5 proved by Lamberton, Pages and 
Tarres in [6] for nonincreasing step sequences (7 n )neN- 

In practice, the Narendra algorithm is used in the context of per- 
formance assessment, in applications either in automatic control or in 
financial mathematics, and the i.i.d. assumption looks rather unre- 
alistic, since the performance depends in general on parameters that 
evolve slowly and randomly in time. The following framework provides 
a possible generalization. 
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Suppose that (S^igx, i G {A,B}, are ergodic stationary Markov 
chains taking values in a measurable space (K,X), with transition 
kernel Qi and stationary initial distribution tt£. Let us consider a 
measurable event C G X, and define sequences (77^)^, for £ G {^4, -B}, 
as follows: 

%,i= (2) 

These random sequences (rj£ are functions of Markov chains and 
satisfy, as a consequence, the ergodic condition (E), with 

8t = irt(Si,o G C). 

The sequences (S^i)^, £ G {A, f?}, represent the agents outputs, 
from which (r/i^ieN extracts scores through target assessment. Note 
that, contrary to (<S^,i)ieN> (%,i)ieN is n °t Markov in general. 

Miao and Yang [7] establish under weak conditions (concerning 
mainly the transition kernels Qi) the law of iterated logarithm for 
additive functionals of Markov chains, thus providing the required 
ergodic rate of convergence (E2). 

Notation: the letter C will denote a positive real constant that 
may change from one inequality to the other. 

We write <j>' and eft for the first and second order discrete deriva- 
tives of <j>: for all n ^ 1, 

(j)'(n) := 4>(n)-4>(n-l), and <j/'(n) := 0(n-l)+</>(n+l)-20(ra). 
We let, for all n G N, 

a n := R n /(p(n), (3 n := supa k . 

Note that, under assumption (E0), a n , n — > 0. 

n— >oo 

Given two real sequences (ii n ) n ^o and (vn)n^(b we write 
when, for all n ^ 0, |u n [ ^ 

1.2 Sketch of the proofs of Theorems [2] and [3] 

Our first aim is to write down, in the following Proposition [H the 
evolution of (X n ) n ^o as a stochastic perturbation of the Cauchy-Euler 
procedure defined by 

x n+1 = x n + y n+ ih(x n ), (3) 
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where h(x) := (9a — 9s)f(x), with f(x) := x(l — x). 

However, contrary to the case of i.i.d. payoff sequences (%, n )n^o> 
t £ {A, B} considered in [6], the perturbation of the scheme ([3]) under 
an ergodic assumption (E) does not only consist of a martingale, but 
also of an increment whose importance depends on cj>, i.e. on the rate 
of convergence of the mean payoffs to 9 a and 9b- More precisely let, 
for all n ^ 1, 

n 

A n = } ]lkf(Xk-l)(r]A,k - VB,k - (9a - 9b)), 
k=l 

with the convention that Ao = 0, and let (M n ) n ^i be an (T n ) n ^i- 
adapted martingale given by 

n 

M n :=Y,lke k , M :=0 

k=l 

with 

£fe :=T] A)k (l-X k -x)( lu k =A-X k -i)+VB,kXk-i((l-X k ^x)- lu k =B)- 



Proposition 4 For all n E N, 

n 

X n = x + M n + A n + (9 a - 9 B ) Jkf(X k -i). 

k=l 

PROOF: The updating rule (pQ) can be rewritten as 

X n +1 = X n + 7n+l^A,n+l(l - X n ) lu n+1 =A ~ 7n+lVB,n+lX n lu n+1 =B 
= X n + J n+ ir] A ,n+lO- ~ X n )( lu n+1 =A ~ X n ) (4) 

+ 7n+lVB,n+lX n ((l - X n ) - Iu„ +1 =b) + ln+lf(Xn)(VA,n+l ~ VB,n+l)- 

□ 

A crucial point here is the estimate the evolution of (A n ) nG N- The 
sequence r]A,k — VB,k ~ (@A — 9b) being erratic by the very nature of 
the question, we would like to use an Abel transform, i.e. discrete 
integration, in order to make the ergodic upper bound function <j> 
appear in estimates. However, (^ n j '(X n -i)) n eN is not a nonincreasing 
sequence in general so that the technique cannot work directly. 
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Instead, let us define, for all n € N, 

A 7w 5 - 1 



with the convention that Ao = So := 1. 

Note that x/S n is a trivial lower bound for X n , and that 



n 



In = 4^, with S n = S^/\ k . (5) 

fc=0 

We first study the sequence (\? n )neN defined by 

OO 

*™ : = E -^ h -{VA,k-VB,k-(0A-0 B ))\ 
(^n)n^l is well-defined since, for all £ G {^4, 5} 

OO OO OO 



Ik 

Sk 



fc=2 K 1 fe=2 K 1 ' x fc=2 

1 " 71 ^ Si ~ 1 - 71 f-i Si ^ 1 - 71 is 



" A fc 1 ^ S fe - 5 fe _i „ 1 /- 00 = 1 1 
fc=2 - - /i fc=2 1 " 71 J Sl 1 " 71 ft 

Since (jn/ S n -i)neN is a nonincreasing sequence if (7 n )neN is itself 
nonincreasing (recall that j n € (0, 1)), we deduce the following Lemma 
[5]by an Abel transform, i.e. discrete integration. Moreover we observe 
that, for all n ^ m ^ 0, the evolution of A. between time steps m and 
n is given by 

n 

Sk-if(X k -i)- — (rjA,k - i]B,k - (6a - B ))- 

k=m+l 

Now, (Skf(Xk))keN is a nondecreasing sequence. Indeed, for all 
k € N, f(Xk) ^ (1 — 7fc)/(Afc_i) since / is concave and is the 
barycentre of X^-i and either or 1, with weights 1 — 7fc and 7&, 
where /(0) = /(l) = 0. We rely on this monotonicity and apply an 
Abel transform again, which enables us to show the following Lemma 

El 

Lemma 5 Assume that (7 n ) n eN is nonincreasing, and that <j> is non- 
decreasing concave on [ko, oo) for some ko E N. Then, for all n ^ ko, 

l*n|^|^ L [<A'(n)+27n0(n)]. 



< OO. 
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Lemma 6 Let, for all n G N, 

2sup k>n p k [<t>'(k)+2j k <j>(k)] 

R n '- ' 

In 

Under the assumptions of LemmalE we have, for all n ^ m ^ ko, 



lkf{X k - l ) + 2f{X n ) 

k=m+l 



A n - A m | ^ R[ 



Lemmas \5\ and O are proved in Sections 13.21 and 13.31 

These results enable us to conclude the proof of Theorem [2j In- 
deed, by Proposition H] and Lemma [6l for all n m ^ 0, 

n 

X n -X m = M n -M m + A n -A m + (8 A -9 B ) Yl 1kf{Xk-\) 

k=m+l 

(6) 

n 

= M n -M m + (9 A -6 B + n(R' m )) lkf(X k - 1 ) + 2U(R' m )f{X n ). 

k=m+l 

We assume that (El) and (SI) hold; thus R' n — ► 0. Since (M n ) ra >i 

n— »oo 

converges a.s. by Doob's lemma (using that ^ 7^ < 00, by Lemma 
[I]), and X n G [0, 1] for all n G N, we deduce that 



^2lkf{Xk-i) < 00 a.s., (7) 



fc=l 

which subsequently implies that, P^-almost surely, (X n ) n ^Q is a Cauchy 
sequence and therefore converges to a limit random variable X^, G 
[0,1]. Now <[7|) implies that f(X OQ ) = 0, since T n — > 00, and there- 

n— >oo 

fore Xqo = or 1 a.s. 

The proof of Theorem [3] itself has two parts. The first one con- 
sists in showing a "brake phenomenon", i.e. that (X n ) n ^Q cannot in 
any case decrease too rapidly to as n goes to infinity. We already 
observed that, trivially, X n is lower bounded by x/S n . A better lower 
bound can easily be obtained: let us define, for all n G N, 

Sjf := =fi — 7 o with initial condition Sq = 0, 

llfe=l(l - 7fc *-{I k >X k> VB,k=l}) 



<s 



and, for all n ^ 1, 

a B c-B vB pBv 

AA n .— 7 n O n , I n .— O n v\ n . 

Note that, as a consequence of the definition of the Narendra al- 
gorithm dU), for all n 0, 

V B _ \ Y n + A n+i(! " *n) if U n+1 = A and ^,n+l = 1 , Q , 

y n+l - S W 

! y„ otherwise. 

Roughly speaking, is the product S n restricted to playing and 
winning with B; x/S^ is straightforwardly a lower bound of X n . 
Proposition UJ proved in Section 14. Q further claims that, for any 
C > 0, C log S^/Sn is an asymptotic lower bound of X n a.s. on 

{^oo = 0}. 

Proposition 7 Under assumptions (S) and (E2), 



< lim X n = > C < lim sup 



log S*/S% 



oo > r T -a.s. 



The second part of the proof of Theorem [3] assumes 9a > 6b, and 
is given in Section [4.2i Recall that, by Theorem [21 X n converges a.s. 
to or 1 (using the remark that (S)-(E2) implies (El), see remark 
before the statement of Lemma [T|) , so that we only need to show that 
P(limX n = 0) = 0. 

We study (X n ) n -^o as a perturbed Cauchy-Euler scheme and prove 
by Doob's inequality that, starting from C log / for sufficiently 
large C > 0, X n remains bounded away from with lower bounded 
probability, which enables us to conclude that X^ ^ a.s. 



2 Deterministic estimates on the step 
sequence 

We first recall below the two following preliminary remarks in [6j that 
(S2) implies on one hand that YlnLi In < 00 an d ° n the other hand 
that T n — log S n converges as n goes to infinity. 

Then we prove Lemma[T]that (S) implies explicit asymptotic upper 
bounds on (7„)„ e N and (r n ) neN . 
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Preliminary remark 1. Assumption (S2) implies X^nLi 7n < 00 
since, for all n 6 N, 

n n 



k=l 



k=l 

«S C / ue~ 0BU du^C 







ue 



~ dBU du < 00, 



using that u \—> ue~® BU is nonincreasing for u > Q~q ■ 

Preliminary remark 2. The partial sums S n and T n satisfy for 

every n ^ 1, 



i og s n - V < r n < io g s n . 



fc=l 



7fc 



(9) 



This follows from the easy comparisons 

r ' Sn du 



k=l 



Sk 



1 u 

n 



log fiy, 



k=l 

^ logS^-^- 

fe=l 



Si 



s k 

n 9 

7fe 



fc-i 



du 



Ik 



PROOF of Lemma[T} Assume that, for some n € N and A > 1/0b, 
7 n n/logn ^ A. Thus, for all 1 ^ k ^ n, 7& ^ ^4 log n/n since (7 n )n>l 
is nonincreasing; hence T n ^ Alogn. Subsequently 

7n = 0(r n e- esr ") = 0( y Alogne- 9BMogn ) = 0(Alogn/n dBA ), (10) 

where we use that the map x 1— > xe~ dBX is decreasing on [1/#b,oo). 
Identity ()10p yields a contradiction if n is sufficiently large, since A > 
l/9 B . 

Let us now prove the second statement. Note first that assumption 
7 n = O(r n e~ 0srn ) implies that, for all rj < 6b, 7n ^ e _r?r ™ for large 
enough T n . Let 5 n := e r "; then, for sufficiently large T n , 

8 n+l -8 n < e ^+e- r "_ e r„ 

! ~" r " - l) < 2e r "( 1 ^ = 25\-\ 
Therefore, if no is sufficiently large, for all n no, 

^n = v[ Sn ^ + 61 < £ + ^ < 2(n - no) + ^ . 
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This yields, for large n € N, 

T n n ^ log n + C, 
which enables us to conclude. 

3 Abel transforms 
3.1 Preliminary estimates 

The following Lemmas [8] and [9] estimate the error in replacing the 
payoffs r\i^ by their "average success rate" 0£ in a sum weighted by 
a decreasing sequence (£, n )neN, by the use of Abel transforms, i.e. 
discrete integrations by parts. More precisely let, for all n G N and 
£ G {A, B}, 

n 
k=l 

be the corresponding deviation. Lemma [8] upper bounds \^ e n ^ — ^ e m ^\ 
for all n ^ m, whereas Lemma [9] shows that converges to a finite 
value under certain assumptions, which are fulfilled for instance when 
f := 7 and (S)-(E2) holds. 

Lemma [8] is the main tool in the proof of Lemmas [5] and [61 and 
the second part of Lemma [9] will be useful in the proof of Proposition 
[7] providing "brake phenonemon" bounds. 

Lemma 8 Let (£ n )neN be a positive real-valued nonincreasing sequence. 
Assume 4> is nondecreasing on [&o,oo) for some ko G N; then, for all 
n ^ m ^ ko, 

l<€ - <d < An ( E + ) • 

\fc=m+l / 

n 

proof: Let, for all n G N and £ G {^4,-B}, := ^(r#,fc - 0*). If 

fc=i 



11 



n ^ m ^ ko, then 

n 

k=m+l 

n n n— 1 

fc=m+l fc=m+l k=m 

n-1 

Now, using that (£n)n^o is nonincr easing, 
n-l 



$^(&-&+l)«fc 



k=m 



n—1 n—1 

< ^ - £k+i)Rk =^2(£k- ik+i)ak<t>{k) 

k=m k=m 
n—1 / rt— 1 n \ 

< /9m E & " &+lM*0 = An I E ^^^) " E " l ) 

k=m \k=m k=m+l J 



= An J] & ^ " ^ " ^ + " • ( 12 ) 

\fe=m+l / 

In summary, and (I12p imply 



^/3 m J 6(0(*)-0(*!-l)) + 2e^H 

\fc=m+l 

= 0m ( Zk(f>'(k) + 2U4>(m)) . 

\k=m+l I 



□ 



Remark 3.1 Under assumption (E2), i.e. when 4>(k) := fc(log(fc + 
2))~( 1+e ) for some e > 0, then 

^ v 7 (log^ + l))^' 

Indeed, for all x £ M + , 

d<A , , 1 (l + e)z 
(x) - 



dry (log(a; + 2)) 1+£ (a; + 2)(log(x + 2)) 2+e ' 

12 



and 



0'(A;K sup (^)(x). 

x€[k-l,k] \ ax J 



□ 



Lemma 9 Given a positive real-valued nonincreasing sequence (£ n )neN 
let, for all n € N, H n := ^Zfc=i£fc- If 4> is nonincreasing on [&o,oo) 
for some ko € N, YlkLi ^k\<P"(k)\ < oo and limsup neN E n \4>'(n)\ = 
then, for all I € {A,B}, ($^) n6 N converges to a finite real value as 
n goes to infinity. 

In particular, under assumptions (S) and (E2) ; for alii £ {^4, B}, 
($^ i7 ) neN and (*^ i7 / r )neN (whereby = (7„) n eN and^/T = (7„/r n ) neNy ) 
converge to a finite real value as n goes to infinity. 

PROOF: For all m, n ^ ko with n ^ m, Lemma [8] implies 



\fc=m+l 



But 



n-l 



k=m+l k=m+l k=m+l k=m 

n-l 

= ^ H fc - ^(fc + 1)) - S m 0'(m) + H n <£'(«) 



n-l 



£ E k cf)"(k) - E m ^'(m) + (n). 



Let us now prove the convergence of (& e n ^) n( zfq, under assumptions 

(S)-(E2). Then T n = O(logn) by Lemma[H and 4>'(n) = o ( — !— 

\logn 

(see Remark [3.ip . so that T n cj)'(n) — > 0. Now, there exist A, fi G (0, 1) 

n— >oo 

such that 



i"(k)\ = |((6(t + 1) - 4>(k)) ~ WW - 4>(k - 1))| 



_^ +/i) __r (fc _ A) 



^ 2 sup 

xe[k-i,k+l] 



\dx 2 
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and 



3)m 



l + e 



(x + 2)(log(x + 2)) 2 + £ 



-2 + 



1 + 



2 + e 



O 



x(log(x + 2)) 2+£ / ' 



x + 2 V log(x + 2) 
x G M+ \{0}. 



so that Yl^kft'ik) < oo, and the assumptions of the first statement 
are fulfilled. The convergence of (3>„ 7 / r )neN follows similarly, since 

7 „/r„ = o( 7n ). □ 
3.2 Proof of Lemma [5] 

Recall that = (see first paragraph after the definition of ( x I / n)raGN, 
Section [L~2]) . Hence, using Lemma [HI 



Y] 77~~" (j}A,k ~ r} B ,k -(Pa -9b)) 



fc=n+l 



E ^'^ + 2 ^W 



,fc=n+l 



fc=n+l 



7n 



4>(n) 



(13) 



where we use the concavity of <j) in the last inequality. 
Now 



oo oo a oo Q „ 1 

E lk _ \ ~* £±k_ _ \ ^ Jfc — Jfc-1 < 1 



S 2 ^— ' £ 2 

k=n+l " K k=n+l k k=n+l k 



s„ 



so that inequality (|13j) implies the result. 
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3.3 Proof of Lemma [6] 

Note that 

n 

S k -if(Xk-i)-z — (VA,k - VB,k - (6a - B )) 

k=m+l 
n 

k=m+l 
n 

= Yl ^k(S k f(X k ) - 5fc_i/(X fc _i)) + V m S m f(X m ) - V n S n f(X n ) 

k=m+l 

(14) 

Recall that OSfe/(-X"fc))fceN is a nondecreasing sequence (see last 
paragraph before the statements of Lemmas [5] and [6|) , so that (Q2 
implies, together with Lemma [SJ that, for all n ^ m ^ feo> 



A n — A m I < R' 



-it™. 



-it™ 



£ Sfc/(Xfc) ~^ x ^ + /(X m ) + f{X n ) 

.k=m+l 

n 

£ [/(**) " /(^fe-i) + 7fc/(*fc-l)] + f(X m ) + f(Xn) 

,k=m+l 
n 

J2 lkf{X k - 1 ) + 2f{X n ) 

_k=m+l 



4 Proof of Theorem 13 



4.1 Brake phenomenon bound: proof of Propo- 
sition [7] 



Assume that (S) and (E2) hold. Let 



A := < limsup 



Y B 

n 

log S% 



< oo 



j n { lim X n = o) . 



In order to prove Proposition [7J i.e. that P(«4) = 0, we first upper 
bound in Lemma [lOl Then we show that Y? — > oo a.s. on A in 

n— +oo 

Lemma[TT]so that, for every A > 0, X n > X/S^ for large n e N. Both 
Lemmas are shown in Section I4.1.1) we finally conclude in Section 
14.1.21 that A almost surely does not occur. 
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4.1.1 Brake phenomenon: preliminary estimates 

Lemma 10 Under assumptions (S)-(E2) ; there exists L > such 
that, for all n € N, S% < Le dBFn a.s. 

PROOF: Recall that (S) implies ^7n < 00 ( see Prelimary Remark 1 
or Lemma [1]), so that there exists K > such that, for all n £ N, 

s$ ifexp ^ 7fe l {/fc>Xfc , r, B , fc =i}j a.s. 
Now observe that 



Tl 



J2^ k l{i k >x h , VBik =i} = GbTu + Y,^(riB,k - B ) - J^TMb,* 

k=l k=l k=l 

n 

= 9 B T n + - lkVB,k !{4<x fc }, (15) 

k=l 

which enables us to conclude since <3?^ 7 converges to a finite value by 
Lemma [9J □ 

Lemma 11 Under assumptions (S)-(E2), A C {limsup n ^ oc = 
oo} ¥ x — a.s. 

PROOF: On A, there exist L, V > such that, for all n € N, 

^ ^ L'e'^S* < LL', (16) 

r n +i r n 

where we use (S2) in the first inequality and Lemma [10] in the last 
one. 
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Now 



|limsupy n B = oo| = ljr(Y k B +1 -Y k B ) = oo 



.k=l 

d \y k+i k =00 
" \k Tk J 

= l{U k+1 =A}VA,k+l = 00 

- A n |£ ^r^" 1 V^m* = 00 1 

= A n < 2^ p ^.fe = 00 f 

U=i k ) 
( 00 



We use X n — ► a.s. on A (and 7 n — > 0) in the second inclusion, 

n—>oo 

whereas we apply conditional Borel-Cantelli lemma (see for instance 
[2], Theorem 2.7.33) in the second equality, which claims, given a fil- 
tration F = (^ r n ) ng N and an F-adapted bounded real sequence (£ n )n^o 
(i.e. 3M > s.t. i n < M a.s.), that 



IneN J IneN 



OO 



Here £ n := ^nS 3 ^ l{ Un=A }ilA,n/Tn is bounded, using ([16]). The last 
inclusion makes use of S B X n > x for all n e N. 

Now Yl^kilA,k/^k = co a.s. on .4, since on one hand 



and, on the other hand, 



<7/r :=Ef (r?A ' fc " 0A) 
fe=i fc 



converges (deterministically) to a finite value by Lemma [9j □ 
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4.1.2 Proof of Proposition [7] 

We assume that on the contrary ¥(A) > 0, and reach a contradiction 
by proving that limsup rwoo Y** / log(S,f ) = oo a.s. on A. Note that 

n-l 

Yn ■= E A ^+l 1 {/ fc+1 ^ fc }^, fc+ l(l " X fc ) + X 
k=0 

and let, for all A > 0, 

n-l 

^n' := E 7fc+l5 ^ I {-ffe+l<A/S| J } r ?^,*+l) 
fc=0 

%n ,X : = E 7fc + lS ^ min 1, »M,fe+i- 
fc=o V ^ y 

Almost surely on .4,, lim sup n ^ oc = oo by Lemma [TT], and lim n ^oo X n 
limn^oo 7„ = 0, so that 

To show that the right-hand side of this last inequality is infinite a.s. 
on A, we aim to estimate E(Z,f' A ) = E(Z,f' A ) and to upper bound 
K((Zn ,A — Zn' X ) 2 )- In order to yield the latter we first observe that 
there exists M > such that, for all k G N, Tfc+iSf ^ Af ^ MF k , 
by inequality (fT6|) . 
Now 

E((Z* < A - A ) 2 ) 

'n-l 



^Afc+l 



= E (E (7fe+iSf ) 2 min (l, (\ _ min ^ .A 

< Mr„E ^E 7fe+1 ^min ^1, Aj ?M,fe+ll = MT n E(Z^ x ). (17) 
On the other hand, for all M > and e > 0, 

nz^ X ) = E (E^+1 5 ** min (*> ^) ^AJH-iJ 

n-l 

>A(1- £ )P(,4) e 

Tfc+l'M.fc+l, 

fc=fco(e,A) 
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where we use that S B = Y^/X n — > oo a.s. on A, ko(e,\) being a 
constant depending on e and A. Now = Y^k=o lk+iVA,k+i-T n QA 
converges by Lemma [91 so that we obtain 

X9 A > limsup V '- ^ liminf V ' > AP(„4)6> A . 

n^oo 

Fix p £ (0, 1), and let 



R _ I *7-B,A yB,X 

By (|17p and Chebychev's inequality, 



^pE(^' A )}- 



P 2 E (Z^' A 

Therefore, for all A > 0, if we let C\ := A PI {limsup^^^ B ni A}, 

P(C A ) > lim sup n B,,,) > P(-4) - > 0, 

if we choose A such that A > M9 A ^{pW(A)y 2 . 

Now, almost surely on C\ C „4, Z,f' A /T n — ► A/9,4 (since 5^ — > oo, 

n— >oo n— >oo 

see above), so that 

limsup- -g ^ , 

n -*oo log b% Ob 

using that limsup n ^ 00 log S B /T n ^9 B by Lemma M 
Therefore 



log S% 



M 



P ( <J limsup 1 y " oR = oo }> n .4 ) ^ P [ limsup C 

. AeN, A^oo 



-A 



> limsup P(C A ) ^ P(.A), 

AGN, A^oo 



which enables us to conclude. 



4.2 Conclusion of the proof of Theorem [3] 

Let, for all n ^ 0, I^f := e dB ^ n . It follows from Proposition [7] that 

limsup - — ~ J? . r> = oo a.s on = 
„^oo log T»/T* 
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using that limsup,^^ /T^ < oo by Lemma [TOl 

Given I € N, let us estimate F(X 00 = 0|^j). Using identity © and 
the assumption 9 a > 9b, there exists uq G N deterministic such that, 
for all n ^ m ^ no, 

n 

X n -X m = M n -M m + (^-^ + n(i?; n )) ^ lkf{X k -i) + 2D( J R , m )/(X r , 

fc=m+l 



^M n -M m -X ni 



so that 



2X n ^X m + M n -M m . 



(18) 



Let (N n ) n ^i be the (^ r n )n>« adapted martingale given by 

n 

iV n := ^ 7 i lix^X^i, Ni := 0; 

recall that (ei)ieN was defined before the statement of Proposition 01 
Let no be sufficiently large, so that j no ^ 1/2; then, for all n ^ no, 
X n+ \ > X n /2. Thus, for all n ^ Z ^ no, inequality (fT8|) implies 

2X n > X m + iV n - iV m > + iV n - iV m , (19) 

where m := max -{7 ^ i ^ n : X{> Xi/2}; indeed, if n < m then, for 
all m ^ k ^ n — 1, X^+i ^ X[/2, hence X/u ^ Xi, and (|19p trivially 
holds in the case n = m. Hence if x~ := max{— x,0) denotes the 
negative part of x, then 

(2Xoo - Xi/2)- sC sup \N n - N m \ ^ 2sup \N n - Ni\. 

Therefore, by Chebychev's inequality 

4E[[(2X OD -X l /2)-} 2 \T l ] 



PpToo = 0|^) ^ 
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E[ su Pn ^(iV: ra -^) 2 |.F ; | 



(20) 



Now observe that, for all k e N, E(4 +1 |^" fc ) ^ X fc , so that 

Doob's inequality implies 



E 



sup{N n - Ni) 2 ]^ 



n>l 



<4E MT 7' I{x„_ lW /(*»-0 

\n=/+l / 
oo 

< 4X, ^ 7n- 



(21) 



n=i+l 



20 



Let us upper bound YliL n +i 7i m terms of T n . For sufficiently large 

jfe G N, 

T k B +1 - T k B = e '* r *+i(l - e -^7 fe+ x) j> T k+i e B7k + i 
and, on the other hand, by assumption (S), 

Hence, if I 6 N was assumed sufficiently large, 

\ogT B ^ /-logt ^^logT, 

n=/+l n=Z+l 



, — ; i i . — / i 1 v n / "/T, 



(22) 

In summary, it follows from identities (|20p — f)22|) that 

loe T B 

P(X oo = 0|^)^C^-. 

Now the bounded martingale P(X QO = 0|^) converges, as I goes to 
infinity, to 

I {w) < C li e mf^ = „. 



so that P(Xoo = 0) = 0. 
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